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Abstract 

Mathematical reasoning involves comprehending mathematical information and concepts in 
a logical way and forming conclusions and generalizations based on this comprehension. 
Computer-based learning has been incorporated into classrooms across the country, and 
specific aspects of technology need to be studied to determine how programs are influencing 
student reasoning and learning. This article explores how one aspect of computer-based 
learning, electronic support tools (ESTs), influences students' mathematical reasoning over 
the course of an online supplemental mathematics program, the Math Learning Companion 
(MLC). Students in Grades 3, 4, and 5 (N = 31) from two private schools participated in MLC, 
and their reasoning was assessed before and after participating in the program. EST use was 
measured by using frequency counts for each tool. Results describe students' tool use and 
reflect an overall change in their reasoning over the course of the intervention, indicating that 
students use ESTs as needed to individualize the learning program. Students specifically used 
ESTs as needed to improve their mathematical reasoning, their correctness of response, and 
their mathematical explanations of their answers over the course of the intervention. 

Keywords: Mathematical reasoning; Computer-based instruction; Electronic support tools; 
Mathematics education 


Introduction 

Despite being one of the wealthiest countries in the world, mathematics proficiency of students in 
the United States, as measured by the Programme for International Student Assessment (PISA), 
lags behind the performance of students worldwide and has remained stagnant for more than a 
decade (Organisation for Economic Co-operation and Development [OECD], 2013a), leaving 
students unprepared for working in a global economy (Hanushek, Peterson, & Woessman, 2012). 
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The availability of educational resources, including computers and software used for instruction 
along with internet connectivity, explains about one-third of the variance in instruction worldwide 
(Education Superhighway, 2014; OECD, 2013b), implying that students with more access to 
educational resources show higher levels of mathematical proficiency. Although students in the 
United States generally have wide access to educational resources as compared to the rest of the 
world (OECD, 2013b), according to the U.S. Department of Education's National Education 
Technology Plan (2010), "the role technology plays in the nation's classrooms varies dramatically 
depending on the funding priorities of the states, districts, and schools and individual educators' 
understanding of how to leverage it in learning in meaningful ways" (p. 9). Thus, the lack of change 
in mathematical proficiency overall indicates a need to further investigate other constructs related 
to mathematical performance, such as mathematical reasoning, within the context of technology. 
Specific type of technological tools such as electronic support tools (ESTs), which are often 
embedded within computer-based educational programs or curricula, should be explored as well 
as their relationship to improvements in mathematical reasoning. 

According to Kilpatrick, Swafford, and Findell (2001), mathematical proficiency can be described as 
five strands of students' cognitive engagement: conceptual understanding, procedural fluency, 
strategic competence, adaptive reasoning, and productive disposition. Current research focuses 
on interventions that address procedural fluency and strategic competence; however, few studies 
include how students communicate conceptual understanding or adaptive reasoning. Part of 
becoming proficient in mathematics involves the ability to reason, communicate this mathematical 
thinking and engage in the process of constructing new knowledge (National Council of Teachers 
of Mathematics [NCTM], 2000). Furthermore, Dreyfus and Eisenberg (1996), discuss the many 
facets of thought, which are common to implicit and explicit characterizations of mathematical 
reasoning such as analogy, representation, visualization, and reversibility of thought. The 
fundamental processes involved in mathematical reasoning, which include quantification, 
patterning, abstraction and generalization, and representation and translation are required for 
students to obtain a sense and understanding of quantity, amount, or number (Dreyfus & 
Eisenberg, 1996; English, 2004). Therefore, thoroughly examining how students describe this 
process and the manner in which they get there through features of the program is critical for 
identifying and understanding methods in which individuals navigate mathematical problems to 
arrive at a solution. 

Mathematical reasoning can be conceptualized as the ability to understand and make sense of 
mathematical concepts in a logical way in order to form a conclusion or judgment (National 
Governors Association Center for Best Practices & Council of Chief State School Officers [NGA & 
CCSSO], 2010; Merriam-Webster, 2014). According to the National Council of Teachers of 
Mathematics' [NCTM] Guiding Principles (2000), "being able to reason is essential to 
understanding mathematics" (p. 3). Reasoning provides students with the ability to comprehend 
all other aspects of mathematics as individuals recognize that mathematical skills and concepts 
make sense by exploring patterns or regularities, synthesizing information, and providing 
arguments to support their conclusions (NCTM, 2000; Rosenstein, Caldwell, & Crown, 1996). 
However, teachers often underestimate students' mathematical reasoning abilities, and 
subsequently these abilities are underutilized as a pathway to mathematical understanding by 
both teachers and students (Francisco & Maher, 2010). 
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The Standards for Mathematical Practice outlined in the Core Curriculum State Standards for 
Mathematics (CCSSM) emphasize the conceptual understanding of mathematics and 
mathematical reasoning along with procedural fluency and problem-solving, as well as encourage 
the use of technology and tools within the classroom to further mathematical understanding (NGA 
& CCSSO, 2010). The CCSSM process standards are derived from both the NCTM's (2000) process 
standards, which include problem solving, reasoning and proof, communication, representation, 
and connections, and Kilpatrick, Swafford, and Findell's (2001) five strands of mathematical 
proficiency: adaptive reasoning, strategic competence, conceptual understanding, procedural 
fluency, and productive disposition. Given that commonalities exist between the CCSSM and the 
PISA, it is plausible to assume that rigorous implementation to the CCSSM standards, including an 
emphasis on mathematical reasoning and the use of technology in the classroom, will improve 
students' mathematical proficiency related to global standards over time (OECD, 2013a). 

Many technologies (e.g. computer software including gaming or skill-based, web-based programs, 
videos) have been shown to enhance reasoning and higher level thinking (O'Donnell, 2006; 
D'Angelo, Higgins, & Crawford, in press). Kramarski and Zeichner (2001) show how metacognitive 
feedback from a computer-based program can enhance student reasoning in mathematics over 
and above simple result-based feedback. This research replicated an earlier study with the same 
findings in a non-computerized environment (Mevarech & Kramarski, 1997), illustrating the 
influences of metacognitive feedback on mathematical reasoning both within and outside of the 
context of computer-based learning. Yian-Shu, Haw-Ching, Shian-Shyong, and Che-Ching (2014) 
demonstrated the effectiveness of computer-based intelligent tutors on improving students' 
reasoning and problem-solving skills. The programs used in this study gauged the students' 
strengths and weaknesses and adapted to the students' needs. These studies demonstrate the 
ability of computer-based environments to improve reasoning; however, little is known about how 
specific aspects of technology may be directly related to reasoning improvement. 

Many computer-based instructional programs include electronic support tools (ESTs) embedded as 
features of the program that the users must choose to implement (Crawford & Freeman, 2011). 
Most literature on ESTs involves literacy (Anderson-lnman & Horney, 1996; Englert, Manalo, & 
Zhao, 2004); however, students use electronic tools in mathematics as well, such as embedded 
calculators, pop-up mathematical formulas, or audio support. Aleven, McLaren, Roll, and 
Koedinger (2006) found that when students used "hint requests" - a specific type of support tool - 
students' mathematical learning improved over the course of the program. Crawford, Higgins, 
D'Angelo, & Hall (under review) found that the students with lower academic skill scores prior to 
beginning an online program used more ESTs and students with higher academic skill scores used 
fewer ESTs; however, all students showed improvement over the course of the program. This 
demonstrates how students of differing abilities can use ESTs to individualize online learning. 
Furthermore, in a cross-case analysis, Crawford, Higgins, and Freeman (2012) examined how 
students with disabilities maximized the use of an online learning program by tailoring the 
instruction to their own needs through the availability and implementation of ESTs. For example, 
one student diagnosed with dyslexia used an audio support feature throughout the program, and 
another student who had a specific learning disability in mathematics described how the 
dictionary was helpful. These students used different tools to individualize the program to their 
specific learning needs. 
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Given the need to understand how different aspects of technology relate to mathematical 
reasoning and understanding, the current study explores the potential effects of ESTs on changes 
in students' mathematical reasoning over the course of an online supplemental mathematics 
curriculum. This program was delivered through a website and occurred outside of core 
mathematics instruction (whole class, teacher-led instruction). This research seeks to specifically 
answer the following questions: 

1. To what extent are students using electronic support tools to support their learning in a 
digital environment? 

2. In what ways are students' frequency of tool use related to their correctness of 
responses, mathematical explanations, and mathematical reasoning? 


Methodology 


Participants 

Participants included students in third, fourth, and fifth grade (N = 31) and were recruited from 
two private schools located in North Texas. One school focused on students with learning 
differences 1 , and all of the students in the third [n = 7) and fourth (n = 8) grade class participated in 
this study. One of the fourth grade students did not complete the program because he left the 
school; therefore, fourteen students were included from the private school for students with 
learning differences. Furthermore, a general education private school also participated in this 
study. In this school, all but one student in the third grade class (n = 9) participated, and eight 
students in the combination fourth/fifth grade class participated as well (fourth grade, n = 5; fifth 
grade, n = 3). Between both schools and all grade levels, thirty-one students total completed the 
program. 

Overall, eighteen boys and thirteen girls participated in the study. Well over three-fourths of the 
sample were Caucasian and the rest of the sample consisted of African American [n = 3), Asian (n = 
1) and multiracial (n = 1) participants. Slightly more than one third of the sample (n = 11) was 
verified with a primary eligibility category of learning disability, six students were verified with 
'Other Health Impairment' (e.g. ADHD, anxiety), two students were verified with autism, one 
student was verified with a speech/language impairment, and one student was verified with 
emotional disturbance. 

All participants received instruction in English, and only one qualified for free/reduced lunch 
(meaning most students were from a moderate to high socioeconomic status). Teachers reviewed 
student files and reported demographic information through a checklist. 


1 Students with learning differences vary from students with diagnosed learning disabilities, students with 
emotional or behavioral problems, or students who have some other condition that prevents them from 
learning in a traditional classroom setting. 
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Math Learning Companion 

The Math Learning Companion (MLC; Digital Directions International [DDI], 2013) program is 
designed as a supplemental online mathematics curriculum for students in Grades 3-8 who are 
low-performing in mathematics. MLC consists of 73 lessons within seven modules: Math 
Foundations 1, 2, and 3; Number Sense; Algebra; Geometry; and Data Analysis. Students' advance 
through each lesson by completing the following lesson components: (1) Real World (instructional 
set), (2) Vocabulary (introduction of new mathematical terms), (3) Instruction (explicitly delivered), 
(4) Try It (guided practice), (5) Game (independent practice), and (6) Quiz (10-items randomly 
selected that align with lesson content). The MLC curriculum framework is modeled after 
HELPMath © (DDI, 2005), which shows positive effects on an ELL population (Tran, 2005), and in 
2012 met the What Works Clearinghouse criteria without reservations for a single subject study. 
For the purpose of this study, the teachers assigned their class a curriculum sequence of eight 
lessons that closely aligned with the current classroom curriculum. 


Measures 

Math Reasoning Inventory 

The Math Reasoning Inventory (MRI; Burns, 2012) is a formative assessment involving face-to-face 
interviews focusing on core numerical reasoning strategies to assess mathematical reasoning. This 
measure was chosen because it is one of the few measures available that examines mathematical 
reasoning aligned the work of Kilpatrick et al. (2001) and the CCSS, which was used as a resource 
for considering the conceptualization of mathematical reasoning. The theory behind the MRI was 
related to two main dimensions content and cognitive engagement. Cognitive engagement is 
considered as essential dimension for mathematical proficiency as noted in the work of Kilpatrick 
et al. (2001) and is discussed in detail in the technical manual (Bernbaum Wilmont, 2012). It is 
represented in the MRI as "the process through which students are expected to strategize, 
rationalize, and reason" (Bernbaum Wilmont, 2014, p 4). 

The Whole Numbers portion of the MRI was used for this study; this consisted of 10 items in which 
participants were provided a problem, asked to answer the problem without the use of pencil and 
paper, and then asked, "How did you figure this out?" Researchers recorded students' verbatim 
responses, and these responses were used to analyze participants' reasoning skills. The Whole 
Numbers test reliability includes a Cronbach’s alpha of .81, and the individual questions have 
Point-Biserial correlation coefficients ranging from .38 -.53 (Bernbaum Wilmot, 2012). The MRI is 
normed for students in the 4 th through 8 th grade; although we used data from students in Grade 3, 
we verified with the teachers that the content covered in the Whole Number portion of the MRI 
had been covered at some point during the academic year. Thus, it was given across the grade 
levels of this study. Before administering this measure, interventionists watched instructional 
videos on administering the MRI available through the MRI website, after which practice sessions 
of delivering and scoring the MRI were conducted. 
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EST Use 


The specific ESTs within the MLC program include: a calculator, a key terms dictionary, a link to 
formulas, audio support, a 'need more help' button, a notepad, and a social media wall. The 
calculator assists with basic algorithmic functions, but has no scientific or graphing capabilities. 
The audio support will read aloud all of the text on the given screen for every page of MLC (the 
lessons, quizzes, within the dictionary, etc.) in both English and Spanish. The key terms dictionary 
provides definitions for all mathematically-related terms in MLC; for example, if the student either 
goes into the dictionary itself or clicks on the hyperlink for the term 'real numbers', they are 
provided with the definition "All natural, whole, integers, rational and irrational numbers, put 
together in one set." Students are also provided this definition in Spanish, a visual representation 
of 'real numbers', and the ability to have this definition read aloud in either language. Within the 
definition, the terms 'integers', 'rational', 'irrational', 'numbers' and 'together' are also 
hyperlinked, giving the student the opportunity to explore further definitions at the click of a 
button. The formula button provides information on all geometric and algebraic formulas along 
with a visual representation and hyperlinks to the dictionary. For example, if a student needs the 
formula for the area of a rectangle, they are provided with the formula length x width, a picture of 
a rectangle that has 'length' and 'width' denoted, and hyperlinks to the definitions of 'length', 
'width', and 'area'. The need-more-help button appears on pages where students must answer 
questions, and provides the students with more information about the particular concepts 
presented on that page. 

Two of the tools provided are digital writing and communication tools: a notepad and a social 
media wall. A digital notepad gives users the opportunity to take notes directly in the program and 
use these on subsequent pages or the quiz (see Figure 1 for a sample notepad entry), and a social 
media wall allows students to communicate with one another via a blog (see Figure 2 for a sample 
wall post). Both of these features allow students to either type, write, draw, and/or insert 
mathematical symbols into their notes or posts. Frequency counts were recorded and downloaded 
daily each time a student clicked on an EST. 



A fraction is a number that describes part of a whole, 

Figure 1. Example of a Student's Entry in the Digital Notepad Using the Typing and Drawing 
Features 
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Math Foundations 2 , L3 Negative: 


0 isn't a positive or negstive 


■fide Comment Save To Mv N 


May 09 2013 07:52:27 Ah 


I think it is a mix 


Figure 2. Example of a Student Posting a Statement to the Digital Media Wall ("0 isn't a positive or 
negative") and Another Student's Comment on Their Post ("I think it is a mix") 


Procedures 

The Principal Investigator (PI) conducted a structured training session for three researchers that 
included an overview of the goals of the study, specific directions for working with students, 
opportunities for practice administering measures, and delivering the intervention script. The PI 
conducted random field observations of all researchers to ensure the fidelity of the 
implementation of the program. 

MLC is intended to be a supplemental curriculum; therefore, researchers provided the teachers (N 
= 4) information about each lesson's content. Each teacher selected eight lessons and placed them 
in order to best align with their mathematics curriculum over the course of the program, which 
spanned 12 weeks during the second part of the academic year. Two third grade classes 
completed the same lessons from Math Foundations 1, although they were completed in different 
order based on the class's curriculum. One fourth grade class completed a combination of lessons 
from Math Foundations 1 and 2 with varying order, and one combination classroom (five fourth 
and three fifth graders) completed lessons from Math Foundations 1, 2, 3, and Algebra. See the 
Appendix A for a complete list of lessons in order for each class. 

All students were assessed on the Math Reasoning Inventory (MRI) before the program was 
introduced and once the program was completed. Before beginning the Math Learning Companion 
(MLC), students were provided a scripted training on the use of MLC and individual electronic 
support tools (ESTs). Researchers demonstrated the functions and use of each EST for participants 
prior to entering the intervention, and students were then given time to practice using each ESTs 
before engaging in actual lessons. Pretesting and training lasted approximately one week. 
Participants used MLC twice weekly for 45 minutes and completed a total of 8 MLC lessons outside 
of their normal mathematics class time for a total of ten weeks. It should be noted that two 
students did not complete the entire 8 lessons; however, they completed the allotted amount of 
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time for the intervention and could not finish due to the school year ending. One week was 
allotted for post-testing at the end of the year for a total of twelve weeks. 


Data Analysis Plan 

Before data analysis of the MRI could occur, data needed to be organized and coded. Three steps 
were implemented to prepare and code participant data. First, researchers developed a coding 
dictionary for each of the three components of the MRI (Student Answer, Student Explanation, 
and Student Reasoning). Student Answer focused on the correctness of the answer and 
represented four distinct categories - correct, incorrect, self-corrected, or did not answer. Student 
Explanation reflects categories of students' procedural reasoning strategies previously established 
by Burns (2012) that are derived from how participants verbally responded to "How did you figure 
that out?" (Bernbaum Wilmont, 2012). Twenty-six potential categories existed across the ten 
questions; these responses were subsequently collapsed into four broad categories: 1) used the 
standard algorithm, 2) used a method specific to the problem other than the algorithm, 3) gave 
other reasonable explanation, or 4) guessed, did not explain, or gave faulty explanation 
(henceforth simplified to 'guessed'). See Appendix B for definitions and examples of each type of 
Student Explanation. 

Although the categories of students' reasoning strategies capture students' procedural 
competence, the measure did not fully capture students' conceptual understanding. Thus, the 
research team created a construct, Student Reasoning, based on the conceptual understanding 
reflected in students' verbatim responses and grounded in the literature pertaining to elements of 
reasoning in mathematics (Dreyfus & Eisenberg, 1996; English, 2004; Francisco & Maher, 2010; 
Kilpatrick et al., 2001). After examining student responses and generating tentative categories, 
mathematics education experts were consulted for feedback and the categories were refined. 
Student Reasoning was coded into seven categories: 1) no attempt at reasoning (the student did 
not answer the problem), 2) guess attempt but incorrect answer (the student answered the 
problem but said "I don't know " or "I just guessed"), 3) guess attempt and correct answer, 4) 
partial attempt but had a reasoning breakdown, 5) complete reasoning with a calculation error, 6) 
complete reasoning and correct answer, or 7) entirely wrong process. See Figure 3 for examples of 
actual student responses for each MRI category and Appendix C for an operational definition of 
each category. 


Type of 
Reasoning 

Question 

Student 

Answer 

Student 

Explanation 

Student Verbatim 

Reasoning 

No attempt at 
reasoning 

7,000 -r 70 

Did not 

answer 

N/A 

1 cannot answer this 

Guess attempt, 
incorrect 

7,000 -r 70 

Incorrect 

(1,000) 

Guessed, did not 
explain, or gave 
faulty 

explanation 

There's 3 zeros and 7, and 
one zero and then a 7, and 
the other digit... 1 don't 
really know the answer, 1 
just guessed 
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Guess attempt, 
correct 

1,000-998 

Correct 

Guessed, did not 
explain, or gave 
faulty 

explanation 

1 don't know 

Partial attempt, 

reasoning 

breakdown 

100-18 

Incorrect (92) 

Guessed, did not 
explain, or gave 
faulty 

explanation 

100 minus 10 is 90 and 90 - 
2, because 8 + 2 = 10, so 90 
+ 2 = 92 

Complete 

reasoning, 

calculation 

error 

99 + 17 

Incorrect (106) 

Used standard 
algorithm to add 

Added 9 with 7, that's 16, 
carried the 1, and 9 + 1 is 

10, and there's another 1, 
so 1 added it 

Complete 

reasoning, 

correct 

15x12 

Correct 

Gave other 

reasonable 

explanation 

1 knew that 12 times 12 is 

144, so 1 added 12 three 
more times to that 

Entirely wrong 
process 

1,000-998 

Incorrect 

(1,998) 

Guessed, did not 
explain, or gave 
faulty 

explanation 

Because when you have a 
zero plus or minus a number 
it is equal to that 


Figure 3. Examples of MRI Responses in Each Category 

Next, training and reliability was established between the coders. Two researchers completed 
reliability on 11% of the sample with an average of 96% reliability, and all disagreements were 
discussed until consensus was reached. Finally, researchers independently coded the remaining 
MRI interviews. If the participant did not answer the question under the first category, the 
remaining categories were not coded. Therefore, a maximum of 310 responses could have been 
coded for this sample at both pretest and posttest. To reiterate, Student Explanation depicts pre¬ 
existing categories created for the MRI that reflect students' procedural reasoning, whereas the 
current research team generated the Student Explanation categories to describe students' 
conceptual understanding. 

Descriptive statistics were assessed for frequency of reasoning categories at pre- and posttest, and 
for EST use over the course of the program. Chi-square analyses were used to determine changes 
in MRI reasoning categories from pretest to posttest, and the effect size phi was calculated for 
each analysis. Mann-Whitney U tests were conducted to determine the differences between 
Grade 3 and Grade 4/5 (data from the three students in Grade 5 were included with Grade 4 
because the students were in the same classroom environment). One-way analyses of variance 
(ANOVAs) were conducted to determine the differences in EST use for the three MRI categories: 
Student Answer, Student Explanation, and Student Reasoning. Effect sizes (eta-squared) were 
calculated for all significant group differences as defined by having a p-value of less than .05, and 
Bonferroni correction was used to conduct post-hoc analyses as needed (Miller, 1981). 
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Results 


Descriptive Analyses of EST Use 

Means and standard deviations of electronic support tool use reflect student tool usage over the 
course of the entire program. On average, students used the audio feature (M = 47.58, SD = 
55.22), calculator (M = 37.84, SD = 22.68), and notepad (M = 33.90, SD = 19.15) most often and 
used the formula (M = 3.27, SD = 5.32) and the 'need more help' button (M = 5.16, SD = 4.70) the 
least. This replicates the findings on frequency of EST use in Crawford et al. (2012), which used the 
same software program as this study. Descriptive statistics for the frequency of clicks are reported 
in Table 1. 


Table 1. Descriptive Statistics for the Frequency of EST use 


EST 

Min 

Max 

M 

SD 

Audio 

0 

207 

47.58 

55.22 

Key Terms 

0 

64 

12.84 

15.21 

Formula 

0 

17 

3.23 

5.32 

Calculator 

5 

114 

37.84 

22.68 

Need More Help 

0 

23 

5.16 

4.70 

Notepad 

8 

94 

33.90 

19.15 

Wall 

0 

34 

13.35 

9.94 

Chi-square analyses 

reveal significant changes 

from pretest 

to posttest for 

Student Answer, 

Student Explanation, 

and Student Reasoning (see Figure 4). 

Over the course 

of the program, 


students were significantly more likely to answer items correctly and significantly less likely to 
answer them incorrectly (x 2 = 89.374, df=9,p< .001; ES = .54, p < .001). For Student Explanation, 
they were more likely to give other reasonable explanations and less likely to use the standard 
algorithm from pretest to posttest (x 2 = 82.007, df = 9,p< .001; ES = .56, p < .001). 

Interestingly, for the category Student Reasoning, significant increases were found for both 'no 
attempt' at explaining reasoning and complete reasoning with a correct response, and guessing 
with an incorrect response also significantly decreased from pretest to posttest (x 2 = 148.462, df= 
36, p < .001; ES = .75, p < .001). Mann Whitney U tests reveal no differences between Grades 3 
and 4/5 on Student Answer, Student Explanation, and Student Reasoning. 
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Student Answer 

*Correct 
*Incorrect 
Self-Corrected 
Did not answer 

Student Explanation 

*Used standard algorithm 
Other method specific to problem 
*Gave other reasonable explanation 

Guessed 
Did not answer 

Student Reasoning 

*No attempt at reasoning 

*Guess attempt, incorrect 
Guess attempt, correct 
Partial attempt, reasoning breakdown 
Complete reasoning, calculation error 
*Complete reasoning, correct 
Entirely wrong process 
Did not answer 



■ Pretest 

■ Posttest 


0 50 100 150 200 


Figure 4. Changes in Mathematical Reasoning from Pretest to Posttest 

Note: *Reflects differences between the expected and observed frequencies that have a standardized 
residual > 1.96, indicating a significant change from pretest to posttest. 


Student Answers and EST Use 

On the MRI pretest, one-way ANOVAs reveal significant differences for all ESTs with the exception 
of the social media wall. On the MRI posttest, however, there were only significant group 
differences for the key terms dictionary, 'need more help' button, and the social media wall. Post 
hoc analyses were conducted for all individual tools when significant differences occurred. Table 2 
reports the ANOVA statistics and effect sizes for these data. 
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Table 2. One-way ANOVAs for EST Use and Student Answer, Student Explanation, and Student 
Reasoning 


Student Answer 3 Student Explanation 5 Student Reasoning^ 



F 

P 

n 2 

F 

P 

n 2 

F 

P 

n 2 

Pretest 

Formula 

5.248 

.002 

.049 

2.198 

.088 

- 

1.763 

.107 

- 

Audio 

4.333 

.005 

.041 

5.555 

.001 

.053 

2.719 

.014 

.057 

Key Terms 

18.754 

.000 

.155 

3.347 

.020 

.033 

.587 

.741 

- 

Calculator 

5.103 

.002 

.048 

1.821 

.143 

- 

2.256 

.038 

.048 

Need Help 

7.705 

.000 

.070 

2.750 

.043 

.027 

2.870 

.010 

.060 

Notepad 

4.891 

.002 

.045 

2.544 

.056 

- 

2.359 

.031 

.050 

Wall 

1.237 

.296 

- 

1.780 

.151 

- 

.794 

.575 

- 

Posttest 

Formula 

1.325 

.266 

- 

.455 

.769 

- 

.398 

.880 

- 

Audio 

1.643 

.179 

- 

1.389 

.238 

- 

2.734 

.013 

.056 

Key Terms 

2.742 

.043 

.026 

2.394 

.051 

- 

1.203 

.305 

- 

Calculator 

2.484 

.061 

- 

3.995 

.004 

.050 

2.490 

.023 

.051 

Need Help 

4.575 

.004 

.043 

4.578 

.001 

.057 

3.884 

.001 

.077 

Notepad 

.063 

.979 

- 

1.354 

.250 

- 

1.281 

.266 

- 

Wall 

5.548 

.001 

.052 

8.175 

.000 

.098 

3.392 

.003 

.068 


Note: The degrees of freedom for Student Answer are 3, 306; for Student Explanation are 3, 298 (pretest) 
and 4, 302 (posttest); and for Student Reasoning are 6, 270 (pretest) and 6, 279 (posttest). 

a Student Answer is comprised of four categories: correct, incorrect, self-corrected, and did not answer. 

b Student Explanation is comprised of four categories: used the standard algorithm, used another method 
specific to the problem, gave other reasonable explanation, or guessed. 

c Student Reasoning is comprised of seven categories: no attempt at reasoning, guess attempt but incorrect 
answer, guess attempt and correct answer, partial attempt but had a reasoning breakdown, complete 
reasoning with a calculation error, complete reasoning and correct answer, or entirely wrong process. 


Post hoc tests reveal that only two ESTs had significant differences on Student Answer at both 
pretest and posttest: the key terms dictionary and the 'need more help' button (see Table 3 for 
means and standard deviations used for post hoc comparisons). At pretest, students who did not 
answer the items were more likely to use the key terms dictionary than students who answered 
the items correctly, students who answered the items, and students who self-corrected when 
answering. At posttest, however, students who answered the item correctly were more likely to 
use the key terms dictionary than students who did not answer the item, indicating a shift in 
correctness of answer over the course of the intervention. Also, students who did not answer the 
items at pretest were more likely to use the 'need more help' button than students who answered 
the items correctly and students who answered the items incorrectly. This difference remained at 
posttest for students who did not answer the item and answering correctly but not for incorrect 
answers. 
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Table 3. Means and Standard Deviations of Tool Use for Each Category of Student Answer 



Correct 

Incorrect 

Self-Corrected 

Did Not Answer 

Pretest 

N = 89 

N = 183 

N = 6 

N = 82 

Formula* 

4.36 (6.65) 

2.33 (4.25) 

2.17 (2.99) 

5.38 (5.17) 

Audio* 

32.79 (47.98) 

55.20(59.78) 

14.17 (13.60) 

51.44 (27.52) 

Key Terms* 

10.85 (13.37) 

10.86 (11.28) 

9.83 (14.13) 

30.22 (24.60) 

Calculator* 

33.21 (20.54) 

37.92 (20.06) 

34.67 (8.89) 

50.81 (34.17) 

NMH* 

4.24 (4.10) 

5.00 (4.03) 

5.33 (4.27) 

8.63 (7.26) 

Notepad* 

34.40(16.86) 

31.95 (17.50) 

26.50 (9.20) 

45.06 (27.72) 

Wall 

13.62 (9.64) 

12.78 (9.98) 

19.50 (9.63) 

14.75 (9.05) 

Posttest 

N = 112 

N = 156 

N = 9 

N = 38 

Formula 

3.21 (5.55) 

3.01 (4.89) 

6.56 (7.35) 

3.42 (5.04) 

Audio 

40.93 (54.65) 

53.33 (58.65) 

25.67 (49.90) 

48.94(22.00) 

Key Terms* 

11.43 (14.68) 

12.41 (13.18) 

12.56 (16.96) 

19.73 (21.28) 

Calculator 

35.64(22.63) 

38.10(21.94) 

29.11 (14.05) 

26.42 (23.44) 

NMH* 

4.43 (4.25) 

5.32 (4.58) 

3.00 (3.39) 

7.48 (5.57) 

Notepad 

34.33 (18.25) 

33.66 (19.68) 

31.89 (11.30) 

34.15 (19.32) 

Wall* 

12.87 (9.79) 

12.43 (9.40) 

12.00 (8.05) 

19.76 (10.11) 


Note: N reflects the number of items or instances. 

*See text for further description of significant post hoc findings. 


Post hoc analyses also indicate that students who answered the items correctly were more likely 
to use the formulas than students who answered incorrectly, and students who did not answer the 
items at all were more likely to use the formulas than students who answered incorrectly. 
Students who did not answer the items were more likely to use the calculator than students who 
answered the items correctly and students who answered the items incorrectly. Also, students 
who answered the items incorrectly were more likely to use the audio than students who 
answered the items correctly. Interestingly, students who did not answer the items were more 
likely to use the notepad than students who answered the items correctly and students who 
answered the items incorrectly. For these three particular ESTs (formulas, calculator, and audio), 
there were no significant differences in tool use on the MRI posttest. On the other hand, there 
were no significant group differences at pretest for the social media wall; however, at posttest 
students who did not answer an item were more likely to use the wall than students who 
answered an item correctly and students who answered incorrectly. 


Student Explanations and EST Use 

For the MRI pretest and Student Explanation, significant differences exist between groups for the 
use of audio and the key terms dictionary (see Table 2). Post hoc analyses reveal that students 
who guessed (to reiterate, this category captures when students stated "I just guessed," "I don't 
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know," or gave a faulty explanation), were more likely to use the audio feature than students who 
used a method specific to the problem or students who gave another reasonable explanation (see 
Table 4). This difference was no longer significant at the posttest, however. For the key terms 
dictionary at pretest, students who used the standard algorithm were more likely to use the key 
terms dictionary than students who used a method specific to the problem and students who 
guessed. At posttest, however, students who did not answer the question were more likely to use 
the key terms dictionary than students who gave another reasonable explanation. 


Table 4. Means and Standard Deviations of Tool Use for Each Category of Student Explanation for 
Post hoc Comparison 



Standard 

Algorithm 

Another Method 
Specific to 
Problem 

Other 

Reasonable 

Explanation 

Guessed 

Did Not Answer 

Pretest 

N = 69 

N = 72 

N = 24 

N = 137 

N = 32 

Formula 

4.04 (5.50) 

3.90 (6.41) 

3.88 (5.54) 

2.40 (4.33) 

5.38 (5.17) 

Audio* 

46.59 (46.76) 

32.19 (46.74) 

28.97 (46.63) 

60.41 (60.98) 

51.44 (27.52) 

Key Terms* 

17.94(20.56) 

11.03 (12.86) 

13.29 (19.05) 

11.58 (11.44) 

30.22 (24.60) 

Calculator 

42.33 (28.99) 

33.63 (19.15) 

35.96 (17.73) 

37.98 (21.02) 

50.81 (34.17) 

NMH 

6.32 (6.00) 

4.39 (3.66) 

3.88 (2.83) 

5.30 (4.55) 

8.63 (7.26) 

Notepad 

39.35 (23.98) 

32.49 (13.39) 

35.17 (19.83) 

31.99 (18.28) 

45.06 (27.72) 

Wall 

15.54(10.52) 

11.78 (8.41) 

13.33 (8.29) 

13.18 (10.29) 

14.75 (9.05) 

Posttest 

A/ = 35 

N = 75 

N = 36 

N = 127 

N = 33 

Formula 

3.14 (5.16) 

2.61 (4.96) 

3.42 (5.76) 

3.24 (5.15) 

3.42 (5.04) 

Audio 

54.69 (60.50) 

41.55 (54.53) 

32.75 (44.03) 

53.06 (60.87) 

48.94 (22.00) 

Key Terms* 

13.26 (12.88) 

11.52 (14.06) 

9.33 (13.31) 

12.66 (14.04) 

19.73 (21.28) 

Calculator* 

45.80(27.77) 

32.88 (20.68) 

31.22 (14.87) 

38.50(22.17) 

26.42 (23.44) 

NMH* 

5.34 ( 4 . 44 ) 

4.79 (4.22) 

2.78 (2.67) 

5.44 (4.82) 

7.48 (5.57) 

Notepad 

39.97 (24.00) 

33.00(17.05) 

35.31 (15.66) 

32.09 (18.83) 

34.15 (19.32) 

Wall 

15.40(11.08) 

12.96 (9.50) 

9.19 (6.87) 

12.05 (9.16) 

19.76(10.11) 


Note: N reflects the number of items or instances. 

*See text for further description of significant post hoc findings. 


No other differences existed at pretest; however, posttests revealed new group differences for 
some of the students. Students who used the standard algorithm were more likely to use the 
calculator than students who used another method specific to the problem. Also, students who 
either guessed or did not answer the problem were more likely to use the need more help feature 
than students who gave other reasonable explanations. As with Student Answer, there were 
significant differences for using the social media wall on the posttest. Interestingly, students who 
did not answer the question were more likely to use the wall than students who used a method 
specific to the problem, students who gave another reasonable explanation, and students who 
guessed. This indicates that students who relied on standard methods to solve a problem were 
using the calculator more frequently, students who did not provide adequate reasoning were 
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more likely to use the help feature, and students who did not attempt the problem at all were 
more likely to use the social media component of the program (see Figure 3 and Appendix B for 
actual examples of student responses). 


Student Reasoning and EST Use 

ANOVAs reveal significant differences between groups for both audio and the 'need more help' 
button at both pretest and posttest (see Table 2 for ANOVA results and Table 5 for means and 
standard deviations used in the post hoc analyses; Figure 3 and Appendix C for operational 
definitions and actual examples of student responses). At pretest, students who guessed at the 
answers and got them correct were more likely to use the audio feature than students who 
guessed at the answers but got them incorrect; however, on the posttest, students who guessed 
incorrectly were more likely to use the audio feature than students who had complete reasoning 
and answered correctly and students who used the entirely wrong process. This indicates a shift in 
student behavior over the course of the program. Also, students who guessed and got the answer 
correct were more likely to use the 'need more help button' than students who guessed and got 
the answer incorrect and students who had complete reasoning and gave a correct answer on the 
pretest. Notably, by the posttest, students who made no attempt at the problem (e.g. refused to 
answer the problem or asked to skip it) were more likely to use the 'need more help button' than 
students who guessed, students who had complete reasoning but made a calculation error, and 
students who had complete reasoning and got the problem correct. 


Table 5. Means and Standard Deviations of Tool Use for Each Category of Student Reasoning for 
Post hoc Comparison 



1 

2 

3 

4 

5 

6 

7 

8 

Pretest 

N = 6 

A/ = 46 

N = 5 

A/ = 51 

N = 52 

N = 91 

N = 26 

N = 32 

Formula 

7.17 

2.54 

4.40 

2.08 

2.42 

4.09 

2.69 

5.38 


(7.89) 

(4.77) 

(6.80) 

(4.02) 

(4.36) 

(6.42) 

(4.25) 

(5.17) 

Audio* 

29.83 

69.83 

28.80 

53.14 

49.42 

31.96 

50.15 

51.44 


(31.26) 

(69.58) 

(17.20) 

(56.15) 

(58.19) 

(47.89) 

(54.62) 

(27.52) 

Key Terms 

13.33 

10.63 

15.20 

9.75 

12.96 

10.01 

10.77 

30.22 


(11.11) 

(10.11) 

(10.40) 

(9.14) 

(14.47) 

(12.55) 

(11.05) 

(24.60) 

Calculator 

32.00 

34.57 

52.20 

34.69 

42.65 

32.67 

38.81 

50.81 


(9.10) 

(16.11) 

(38.96) 

(19.20) 

(19.85) 

(17.87) 

(25.55) 

(34.17) 

NMH* 

3.83 

4.54 

10.40 

5.14 

5.37 

3.90 

5.46 

8.63 


(1.72) 

(3.35) 

(7.80) 

(4.59) 

(3.07) 

(3.58) 

(5.74) 

(7.26) 

Notepad 

41.17 

25.67 

48.20 

35.06 

32.85 

33.23 

33.73 

45.06 

(32.20) 

(14.06) 

(32.38) 

(16.97) 

(17.94) 

(15.22) 

(20.94) 

(27.72) 

Wall 

19.67 

12.30 

16.20 

11.98 

13.42 

13.86 

14.08 

14.75 


(7.63) 

(9.82) 

(10.62) 

(10.86) 

(8.54) 

(9.62) 

(11.34) 

(9.05) 
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Posttest 

N = 16 

N = 35 

N = 6 

A/ = 46 

A/ = 45 

N = 116 

N = 22 

N = 33 

Formula 

3.38 

3.54 

4.67 

3.54 

2.82 

3.36 

1.91 

3.42 


(5.63) 

(5.86) 

(6.68) 

(5.14) 

(4.55) 

(5.69) 

(3.83) 

(5.04) 

Audio* 

51.19 

76.86 

65.33 

45.13 

49.07 

38.18 

30.86 

48.94 


(22.68) 

(77.06) 

(75.12) 

(49.03) 

(55.02) 

(52.92) 

(33.77) 

(22.00) 

Key Terms 

9.44 

13.91 

19.17 

11.85 

14.16 

11.07 

7.45 

19.73 

(4.37) 

(15.65) 

(22.37) 

(10.33) 

(15.33) 

(14.26) 

(5.06) 

(21.28) 

Calculator* 

54.25 

34.80 

46.50 

35.30 

37.98 

34.50 

41.82 

26.42 


(28.34) 

(19.35) 

(34.09) 

(14.97) 

(18.64) 

(21.31) 

(31.06) 

(23.44) 

NMH* 

8.75 

4.54 

8.17 

5.22 

4.58 

4.18 

6.68 

7.48 


(6.95) 

(4.50) 

(7.91) 

(33.34) 

(2.81) 

(3.91) 

(6.80) 

(5.57) 

Notepad 

26.94 

28.89 

27.00 

37.52 

33.69 

34.44 

34.82 

34.15 


(7.35) 

(17.19) 

(25.25) 

(19.26) 

(21.26) 

(17.35) 

(19.72) 

(19.32) 

Wall* 

22.44 

13.46 

17.50 

11.74 

10.84 

12.72 

13,55 

19.76 


(11.24) 

(8.94) 

(10.29) 

(9.79) 

(8.13) 

(9.72) 

(10.08) 

(10.11) 


Note: N reflects the number of items or instances. Student Reasoning is comprised of seven categories: 1) no 
attempt at reasoning, 2) guess attempt but incorrect answer, 3) guess attempt and correct answer, 4) partial 
attempt but had a reasoning breakdown, 5) complete reasoning with a calculation error, 6) complete 
reasoning and correct answer, or 7) entirely wrong process. Category 8 represents unanswered items. 

*See text for further description of post hoc findings. 


Two other ESTs showed significant group differences at posttest but not at pretest: the calculator 
and the social media wall. Students who had complete reasoning and answered correctly were 
more likely to use the calculator than students who made no attempt at the problem. 
Furthermore, students who made no attempt at the problem were more likely to use the wall than 
students who guessed incorrectly, students who partially explained their thought process, 
students who had complete reasoning but a calculator error, and students who had complete 
reasoning and got the correct answer. 


Discussion 

Mathematical Reasoning 

Given the plethora of research indicating the importance of mathematical reasoning (Kilpatrick et 
al., 2001; NCTM, 2000; NGA & CCSSO, 2010), findings from this study are consistent with the idea 
that reasoning itself can be directly taught (Rosenstein et al., 1996) and technologies can be used 
to enhance reasoning (Kramarski & Zeichner, 2001; Mevarech & Kramarski, 1997; O'Donnell, 2006; 
Yian-Shu et al., 2014). Overall, students showed improvement in their answers, explanations, and 
reasoning after interacting with the MLC program. Results denote more correct answers on the 
posttest than the pretest, more students were giving other reasonable explanations, and fewer 
were using the standard algorithm on the posttest, indicating a shift in procedural understanding 
of the mathematical concepts. As anticipated, fewer students were guessing incorrectly and more 
were giving correct responses with complete reasoning on the posttest, indicating an overall shift 
from weaker to stronger conceptual reasoning skills over the course of the program as well. This 
finding is consistent with studies that show different ways that support tools embedded in 
technologies can be used to positively impact reasoning skills (Aleven et al., 2006; Crawford et al., 
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2012; Crawford et al., under review). Unexpectedly, however, significantly more students chose 
not to provide any reasoning on the posttest than on the pretest. This could be due to a lack of 
confidence about their answers that was not present at pretest or an awareness of an inability to 
correctly answer the problem. Students may have learned enough over the course of the 
intervention to know when their reasoning was incomplete; however, they had not learned 
enough to confidently make an educated guess at the answer. These results may indicate that a 
shift in students' thinking occurred over the course of the intervention. 


Relationship of Correctness of Response to EST Use 

Being able to provide correct responses to problems is essential to students' procedural fluency in 
mathematics (Kilpatrick et al., 2006). Within this study, frequency of EST use throughout the 
program reflects how students use the tools to maximize the potential benefits of the program. 
Generally, students who were more likely to give correct answers at pretest were less likely to use 
the ESTs than students who either answered the item incorrectly at pretest or students who chose 
not to answer the item, with the exception of the formulas and the social media wall. At posttest, 
however, students who answered correctly were more likely to have used the key terms 
dictionary. Students who did not answer were more likely at both pre- and posttest to use the 
tools than students who answered either correctly or incorrectly, indicating a possible lack of 
knowledge, reasoning, and/or confidence about answering the question. These findings appear to 
mirror findings from other studies with respect to how and why students access specific types of 
ESTS. For example, Renkl (2002) reported the effects of a "help feature" for solving probability 
problems in mathematics. Those in the treatment condition who had access to the "help feature" 
scored significantly better on the post-test than those in the control group. Additionally, Aleven et 
al. (2006) found that students' math scores improved when they used help tools effectively. 


Relationship of Student Explanation and Reasoning to EST Use 

For Student Explanation, students who gave other reasonable explanations or used a method 
specific to the problem on the MRI were less likely to use tools at both pre- and posttest than 
students who guessed, used the standard algorithm, or did not initially answer the problem. This 
reflects the idea that students who already have procedural understanding of these concepts may 
not need the tools as much as students who are unfamiliar with the mathematical procedures 
(Kilpatrick et al., 2006). The differences in Student Reasoning reveal that students who were 
guessing correctly at the pretest were more likely to use the audio feature and the 'need more 
help' button. At posttest these differences shifted to students guessing incorrectly or making no 
attempt at reasoning. This demonstrates how students with weaker conceptual reasoning skills 
(e.g. making no attempt, guessing) are more likely to use ESTs than students with stronger 
conceptual reasoning skills (e.g. complete reasoning with either a correct or incorrect answer). 

These findings expand on the differences in tool use found in previous research by showing how 
students use ESTs to individualize the program based on their prior ability to reason through a 
problem (Aleven et al., 2006; Crawford et al. 2012; Crawford et al., under review). Students who 
enter the program with weaker reasoning skills, demonstrated by either guessing or not 
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attempting the problem, are more likely to use a variety of ESTs than students who enter with 
stronger reasoning skills. The results also support previous studies that demonstrate how 
technologically-based programs can enhance students' mathematical reasoning (Kramarski & 
Zeichner, 2001; Yian-Shu et al., 2014), expanding on this idea by focusing specifically on EST use in 
mathematics. 


Limitations 

Several limitations for the current study need to be addressed. The sample size was small and 
from two private schools; because public and private schools take different approaches to 
mathematics education, the generalizability of this study is limited. The public school curriculum is 
largely governed by decisions at the state and school district level; however, private schools can 
make curriculum decisions at a building level. Therefore, the curricula that are implemented and 
the nature of the instruction may look very different a public and a private school. Also, students 
did not complete the same curriculum across classrooms and grade levels, thus some of the 
observed effects could be confounded by the curriculum itself. 

Although the MRI is a validated measure, the Student Reasoning category was developed by the 
researchers for this study and is subjective in nature; thus, it needs to be validated across several 
studies. Also, the specific ESTs measured in this study represent only a few ESTs available outside 
of MLC, and (with the exception of the 'need more help' button) were available to the students at 
all times during the program. Therefore, although strong associations are made in this study 
between EST use and reasoning, it is difficult to draw definitive conclusions regarding reasoning 
and the use of ESTs. Finally, other measures of mathematical achievement could provide further 
insight into student's EST usage beyond mathematical reasoning. 


Implications for Future Research 

The current study demonstrates how the use of ESTs can enhance students' mathematical 
reasoning because students are able to individualize a technology-based program to meet their 
own needs. Also, students showed improvement in reasoning across the course of the program, 
which is related to their EST use. The results of this study present a unique contribution to the 
field of educational technology by providing specific details on how students are using specific 
aspects of educational programs to maximize their learning. Students tailor the program to their 
specific needs, which allows for an increase in their mathematical reasoning as well as obtaining 
the correct response. 

These findings lay the groundwork to both replicate and expand upon this study. Future studies 
are needed to validate the Student Reasoning category of the MRI, and future research could 
expand upon this work with a stronger design, such as a randomized control trial, and a larger 
sample of students. Also, research conducted in public schools would be necessary for the 
generalizability of these findings across school settings. The results of the current study provide a 
rationale for examining subgroups (e.g. students with disabilities, students of different levels of 
access to technology at home, students with high versus low computer self-efficacy or reasoning) 
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of students' use of ESTs as well. Because students use ESTs to tailor the program to their individual 
needs, future research could examine similarities and differences across subgroups of students on 
their use of technological tools. Furthermore, individuals who design computer-based education 
programs can use the findings of this study to embed active features that allow students to 
maximize their learning from the program. Because each tool is used differently by different 
learners to enhance their reasoning, the findings highlight the importance of embedding a variety 
of tools within computer-based curricula. 


Conclusion 

The findings from this study emphasize one way in which technology can enhance students' 
mathematical procedural and conceptual reasoning, which has been proven essential for overall 
mathematical proficiency (NGA & CCSSO, 2010; NCTM, 2000). Specific aspects of technology, such 
as electronic support tools, can give rise to changes in mathematical reasoning and furthering 
mathematical procedural and conceptual understanding. By improving mathematical proficiency 
and furthering engagement with technology, students' become more competitive as future 
workers in the overall global economy (OECD, 2013a). 
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School 1 


Appendix A 

MLC Lessons Completed by Each Class 


Grade 3 (n = 9) 

- M FI LI-Place Value 

MF1 L2-Addition & Subtraction 

MF1 L6 - Decimals & Money 

MF1 L3 - Multiplication 

MF1 L4 - Division 

MF1 L5 - Fractions 

MFI L8-Geometry 

MF1 L7 - Measurement 


Grades 4 & 5 (n = 8) 

MF1 L7 - Measurement 

MFI L8-Geometry 

MF2 L2 - Fractions & Decimals 

MF2 L2 - Perimeter & Area 

MF2 L2-Geometry 

MF3 L6 - Division Skills 

MF3 L8-Add & Subtract Fractions 

Alg LI - Interpreting Graphs 


School 2 

Grade 3 [n = 7) 

- MF1 LI-Place Value 

MF1 L2-Addition & Subtraction 

MF1 L3 - Multiplication 

MF1 L4 - Division 

MF1 L5 - Fractions 

MF1 L6 - Decimals & Money 

MF1 L7 - Measurement 

MFI L8 -Geometry 


Grade 4 (n = 7) 

MF1 L3 - Multiplication 
MF1 L4 - Division 
MF1 L5 - Fractions 
- MF2 LI-Place Value 

MF2 L2 - Fractions & Decimals 
MF2 L4-Addition & Subtraction 
MF2 L5 - Multiplication 
MF2 L10 - Perimeter & Area 
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Appendix B 

Definitions of Student Explanation* 

• No Attempt - student does not answer the problem, therefore s/he does not provide any 
explanation 

Example Response: "Can I skip this one?" 

• Used Standard Algorithm - student provides the standard algorithm to add, subtract, 
multiply, or divide when solving the problem 

Example Response: Problem is 1,000 - 998, student responds, "I put the 998 
under the 1,000, and then I cross out the 1 and put a 0, then cross out the next 0 
and put a 9, and cross out the next one and put a 9, and the last 0 
becomes a 10, and then 10 - 8 is 2, and 9 - 9 is 0, and 9 - 9 is 0, so the answer is 

2 ." 

• Other Method Specific To Problem - student uses a strategy to solve the problem 
predetermined by the MRI that is specific for the problem 

Example Response: Problem is 100 -18, predetermines strategies are 1) Added 
up from 18 to 100, 2) Subtracted 20 and then added 2, or 3) subtracted 10 and 
then subtracted 8. Student response, "I did 100 minus 10 which is 90, and then I 
did 90 minus 8 and I got 82." 

• Gave Other Reasonable Explanation - student gave an explanation that provided 
sufficient reasoning but was not listed in the predetermined strategies 

Example Response: Problem is 15 +_= 200, student responds, "I knew that 5 

+ 5 + 5 is 15, so I did 200 - 5 and got 195, minus 5 is 190, minus 5 again is 185." 

• Guessed, Did Not Explain, or Gave Faulty Explanation - student stated that they guessed 
at the response, did not explain their reasoning, or gave reasoning that was incomplete or 
incorrect 

Example Response: Problem is 15 x 12, Possible student responses could be, "I 
don't know, I just guessed." "I have no idea." "I think I do 5 x 2 is 10 and 1 x 1 is 
1 so it is 100." 

*These categories are based on categories previously created by Burns (2012). 
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Appendix C 

Operational Definitions of Student Reasoning Created by Research Team 

• No Attempt - student does not answer the problem, therefore s/he does not provide any 
reasoning 

Example Response: "Can I skip this one?" 

• Guess, Incorrect - students answers the problem incorrectly, but does not provide any or 
adequate reasoning 

Example Response: "I just guessed." "I have no idea." "I think it's 100 minus 
something." 

• Guess, Correct - student answers the problem correctly, but does not provide any or 
adequate reasoning 

Example Response: Problem is 25x4, student responds, "I'm not sure, but I'm 
thinking about 100." 

• Partial Attempt, Breakdown in Process - the student begins to communicate 
mathematical reasoning; however, reasoning is incomplete or incoherent 

Example Response: Problem is 100-18, student responds, "Took away 10 and 
then did 8, 9, 10 so I knew it was 2" 

• Complete, Calculation Error -the student communicates their reasoning coherently; 
however they make a calculation error along the way 

Example Response: Problem is 100-18, student responds, "Because 100-10 is 90, 
and then you take 8 more away, and 10-8 is 1." 

• Complete, Correct -the student communicates their reasoning completely and coherently 
with a correct response 

Example Response: Problem is 60 x 40, student responds, "I took away the zeros 
and did 6x4 and got 24, and then added back the zeros and its 2,400." 

• Entirely Wrong Process -the student communicates their reasoning completely and 
coherently, but performs the wrong operation or describes a logically incorrect process 

Example Response: Problem is 1,000 - 998, student responds, "I added 998 to 
1,000 and I got 1,998." 
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